Skip to content

[do-not-mirror][do-not-merge] chore(fork): scale process cost to change risk - #10

Closed
jdsika wants to merge 3 commits into
integration/3.0from
chore/scale-process
Closed

jdsika wants to merge 3 commits into
integration/3.0from
chore/scale-process

Conversation

@jdsika

@jdsika jdsika commented Nov 11, 2026 •

Copy link
Copy Markdown

What

Makes the fork process cost what the change risks, and reduces the manual steps in it to exactly
one: pressing "Rebase and merge".

The gates stop being rituals and become a check. New fork-only workflow
.github/workflows/fork-gates.yml runs forkctl check-commit plus the upstream-readiness proof
(forkctl mirror <slug> --sha HEAD --no-publish) on every PR into integration/3.0, deriving the
slug from the branch and the chore(fork): exemption from the commit subject. Nobody runs those
commands, and nobody pastes their output into a PR description any more.

Review stops being a separate blocking step. It was: an adversarial review by a non-author, run
as a fresh-context agent for Track A, posted as a PR review listing every probe including the ones
that found nothing
, with each finding resolved in-thread before merge. It is now the merge decision
itself — forkctl track still says how hard to read the diff, PROCESS.md still holds the Track A
prompts, but there is no approval to collect, nothing to post, and nothing written down unless
something was actually found.

Three gates become two checks plus the button. Gate 1 is CI. Gate 2 is Fork gates. The old
Gate 2 (review) is gone as a gate; the old Gate 3 is renumbered.

Three over-strict guards relaxed, all the same bug — conflating "this branch is correct" with
"this branch is current". Each one forced every open PR to rebase and re-run its gates whenever
anything else landed:

  • check-commit: "not rebased directly onto the base" was a failure, now a notice. "Rebase and
    merge" already replays the commit and refuses when it cannot.
  • resolve_feature_commit: required the commit be parented on the tip of integration/3.0, now
    only that its parent is an ancestor of the stack.
  • fork_state_problems: reported a required fork-only file as deleted when the branch simply
    predates it. Now excused only when the file is absent from the branch's own merge base — deleting
    one you did inherit is still caught.

Smaller surface. PR template 54 -> 31 lines, with both gate code blocks and the three gate
checkboxes removed. Section 4.3 inverted: CI runs the suites, you owe pre-commit plus
make rebuild for C changes. Section 8 reduced to five bullets and no commands. CONTRIBUTING.md
and PROCESS.md follow.

New forkctl mirror --no-publish proves the gate without moving mirror/<slug>, so CI cannot leave
mirror branches behind. forkctl delete-mirror is fixed to match: it treated an absent remote branch
as a failed push, so a local-only mirror — now the normal case — could not be cleaned up at all.

Why

Two separate pieces of feedback. The cron jobs had been failing since 29.10 on false alarms, and the
per-PR cost had grown to: 2 gate commands run by hand, 2 pasted output blocks, 4 local suites, 7
checkboxes, a review ceremony with a written probe list, and a forced rebase every time anything
else merged. Most of that is mechanical and was already implemented in forkctl — it was simply
being executed by a person (or, expensively, by an agent) instead of by CI.

The parts that actually need judgement — is this the right change, and can it break the sim — do not
get cheaper by being written down twice. They get cheaper by happening once, at the moment someone
decides to merge.

Notes

The workflow was run (section 4.8 requires this of any PR that adds one):

  • Chore path, this PR: green in 31 s, correctly detecting chore(fork): and skipping the upstream
    half.
  • Feature path, throwaway probe PR [probe] exercise the feature path of the Fork gates check #18: both steps executed and reported from one run
    (if: always()), slug and chore detection correct, each failure carrying its fix. The probe is
    two commits by construction — it sits on top of this unmerged chore commit — so it is a red
    feature path by design.
  • A green feature path cannot be produced in CI until this commit is in the base, because the
    workflow only exists on branches carrying it. Proven locally instead, with this commit's
    forkctl against the real single-commit branch of ci: skip the test jobs on documentation-only pull requests #15:
    mirror ci-skip-doc-only --sha HEAD --no-publish -> OK, clean cherry-pick onto pure
    upstream/3.0, mirror diff one file, branch not moved.

Two bugs the probe found that a review would not have: actions/checkout with ref: <sha> leaves
HEAD detached and mirror refuses to run there (fixed with git switch -c gate-head), and the
parent-on-tip rule above.

Tested: full forkctl suite, pre-commit run --all-files, forkctl check-commit --chore.

Cleanup done alongside this PR (stale state, not part of the diff): deleted the folded
chore/forkctl branch on origin, four orphaned local mirror/* branches with no upstream PR, and
two pre-rebase local leftovers. Every remaining branch now maps to an open PR or is structural.

This is a chore(fork): PR. It is folded into the base commit by fixup and closed unmerged
(section 3.3) — never merged with the button.

Post-fold step, do not skip: add Fork gates to the required checks on integration/3.0. It is
deliberately not required yet — the workflow does not exist until this commit is in the base, and
a required check that cannot run sits pending forever and blocks every merge.

Suggested order (one history rewrite, one rebase per PR):

  1. Merge fix(ci): skip the fused encoder where triton is absent #13 — it clears the triton failure that is red on every other PR.
  2. Fold this PR, rebasing onto origin/3.0 in the same step (closes issue Rebase integration/3.0 onto upstream 3.0 #19).
  3. Add Fork gates to the required checks.
  4. Rebase ci: skip the test jobs on documentation-only pull requests #15, docs: align the comment standard across AGENTS.md and CLAUDE.md #17, fix(eval): resolve carla_cosim benchmarks and lint for unresolved names #14 onto the new tip; they pick up both the triton fix and the gate.

ASCS-eV/PufferDrive is consumed as a git submodule internally while feeding
features back to Emerge-Lab/PufferDrive. This commit carries everything that
exists only in the fork, and is the single place fork-only content lives.

Branch model:
  3.0              verbatim, fast-forward-only mirror of upstream/3.0
  integration/3.0  default branch; this commit plus a linear stack of
                   single-commit features; what internal consumers pin
  feature/<slug>   one feature, exactly one commit
  mirror/<slug>    cherry-picked onto pure upstream/3.0 for upstream PRs

integration/3.0 is the default branch because GitHub reads the PR template,
the contributing guidelines, the agent instructions and scheduled workflows
only from the default branch, and none of those files may exist on 3.0
without destroying the mirror. A clone checks out the default branch, so the
copy of the instructions here is the process as far as every agent and every
submodule consumer is concerned; a rule that lives only in an open PR has no
effect.

Nothing reaches integration/3.0 until three gates pass:
  1. CI green on the final commit.
  2. Adversarial review by someone other than the author, with the findings
     written down -- including the probes that found nothing.
  3. Upstream-readiness proven rather than asserted: the commit cherry-picks
     onto a branch cut from pure upstream/3.0 and produces an upstream-clean
     diff. We stack only commits we could contribute today.

chore(fork): commits are exempt from gate 3, since fork-only infrastructure
can never be upstream-ready. They carry two orthogonal markers: do-not-mirror
(must never reach upstream) and do-not-merge (the merge button is never
pressed -- merging would create a second chore(fork): commit and break the
single-commit invariant that every sync asserts). Such a PR is folded into
this commit by fixup and closed unmerged.

Contents:
- .github/copilot-instructions.md: normative process for coding agents
- CONTRIBUTING.md: human orientation, same rules
- .github/pull_request_template.md: testing and stack-hygiene gates
- .github/workflows/sync-mirror.yml: fast-forward-only mirror sync that
  refuses any non-fast-forward instead of rewriting the upstream base
- AGENTS.md / CLAUDE.md: fenced pointer so agents find the fork process
  without colliding with upstream-owned content
- ci.yml / install.yml: run CI for PRs targeting integration/3.0, which
  upstream's branch filters would otherwise skip

A standing draft PR from integration/3.0 into 3.0 gives a continuous CI
verdict on the combined stack; it must never be merged.

Everything here is fork-only and must never be cherry-picked upstream.
The Linux CI jobs fail while installing the package:

    [Errno 122] Disk quota exceeded

Errno 122 is EDQUOT, hit while unpacking the CUDA torch stack that
`pip install -e .` resolves by default. torch alone is 554.6 MB and the
accompanying nvidia-* wheels add another ~2.4 GB, which no longer fits
the runner's quota. The failure occurs during "Installing build
dependencies", i.e. the isolated PEP 517 build environment is already
enough to exceed it on its own.

The trigger is a runner image change, not a code change:

  * upstream 11_bernhard_dev, 2026-10-20, Image: ubuntu-24.04 -> success
  * same workflows, 2026-10-23, Image: ubuntu-26.04 -> failure

Both resolve an identical wheel set (torch 2.14.1, 554.6 MB). The
`ubuntu-latest` label began migrating to Ubuntu 26 on 2026-10-19
(actions/runner-images#14748), between those two runs. Both workflows
already carry "Free up disk space" steps, so these jobs were running
close to the limit before the migration.

Fix: install CPU-only torch from download.pytorch.org, then build with
--no-build-isolation so the build reuses it instead of resolving torch
a second time. --no-build-isolation needs the build dependencies present
up front, hence the explicit setuptools/wheel/numpy install; the
`setuptools<81` bound matches the pin already in install_requires.

This loses no coverage: BUILD_CUDA_EXT keys off CUDA_HOME/ROCM_HOME,
which is unset on these runners, so the CUDA sources were never compiled
here. tests/smoke_tests/Dockerfile already installs torch this way.

macOS keeps the plain `pip install -e .` unchanged: its PyPI wheels are
already CPU-only, and leaving it alone keeps the default PEP 517
build-isolation path under test somewhere in the matrix.
@jdsika jdsika added do-not-mirror Fork-only change; must never be sent to Emerge-Lab/PufferDrive do-not-merge Never use the merge button; applied by fixup (3.3) or never merged at all labels Nov 11, 2026
@jdsika
jdsika force-pushed the chore/scale-process branch 2 times, most recently from 991016b to 30b1bd3 Compare November 11, 2026 10:05
@jdsika

jdsika commented Nov 11, 2026

Copy link
Copy Markdown
Author

Gate 1 note — the two red checks are not this PR's.

Unit Tests and Notebook Execution fail here on test_drive_mean_action_is_exact_in_physical_units with ImportError: fused_slot_encoder=true requires triton. That is a repository-wide defect inherited from upstream (fused_slot_encoder: true in the shipped config, upstream PR Emerge-Lab#601) and it is red on every open PR, including ones that touch only documentation.

It is fixed by #13, which is now 12/12 green. Once #13 is stacked and this branch is rebased, Gate 1 can be satisfied here.

@jdsika
jdsika force-pushed the chore/scale-process branch 5 times, most recently from 911dd26 to c2b2121 Compare November 11, 2026 13:28
@jdsika
jdsika force-pushed the chore/scale-process branch 3 times, most recently from 4257145 to a8ad898 Compare November 11, 2026 14:32
Two scheduled jobs had been red on every run for a fortnight, and both alarms were false. `sync-mirror.yml` masked a `gh label create` failure with `|| true` and then hard-failed on the `gh issue create` that depended on the label. `fork-only-view.yml` failed its first stack invariant because the stack was one rebase behind upstream, returned early, and so silently stopped running the view-drift and standing-PR checks it exists for.

Both are fixed at the root. The tracking issue is now matched by exact title, so no second object has to exist first, and it is the only alarm for an outstanding rebase. `verify_stack` gains `behind_is_notice`: a behind stack is reported as a warning and the remaining invariants are measured against the mirror the stack was built on, so a sync state can no longer disable a different detector.

The rest reduces what every agent session costs. The always-loaded instructions drop from 6056 to 3941 words, with the rationale moved into PROCESS.md where it is read on demand. Review is now tiered: `forkctl track` classifies a diff as A (pufferlib, config, pufferl.py, models.py, setup.py, smoke goldens) or B (everything else), and only Track A owes a fresh-context adversarial review and the simulator suites. Nits are stated as a process bug, not a review comment. The PR template drops from 85 lines and 20 checkboxes to 56 and 7, by deleting everything forkctl check-commit already proves.
@jdsika

jdsika commented Nov 11, 2026

Copy link
Copy Markdown
Author

Folded into the fork base and closed unmerged, per section 3.3.

  • New fork base: 747f2ae7a33d0f692c2dddeb62650233322ae1a3
  • New integration/3.0 tip: cef3a07a8daef8ab46dea66c36622e1074657332
  • fork-only/3.0 republished to the new base in the same atomic push.

The stack was rebased onto origin/3.0 (73ffdcbb) in the same operation, so this also clears the outstanding upstream-sync debt.

Verification before the push:

  • forkctl check-stack: 13/13 PASS, both before and after the fixup
  • make test: test-unit 332 passed, test-eval passed, test-c passed, test-notebooks 6/6 passed
  • make rebuild: clean

Note for anyone reproducing this: 13 unit tests failed on the first local run with ImportError: pyarrow. That is a stale virtualenv, not a defect -- pyarrow is declared in setup.py and CI installs it via pip install -e ..

@jdsika jdsika closed this Nov 11, 2026
@jdsika
jdsika deleted the chore/scale-process branch November 11, 2026 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge Never use the merge button; applied by fixup (3.3) or never merged at all do-not-mirror Fork-only change; must never be sent to Emerge-Lab/PufferDrive

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant